Purpose of the study: The aim of this study is to test whether switching the verbal orientation of scales and whether the choice of a starting point of scales with numeric labels affect the respondents’ answers.
Design/methodology/approach: The study consists of two split-ballot experiments conducted within the probability-based mixedmode general population panel. The first experiment tests the effects of switching the direction of the scales. The second experiment tests scale labeling options: (1) verbally labeled end points vs. (2) end-point and numeric labels starting with 0 and (3) end-point and numeric labels starting with 1.
Findings: For the first experiment, we find that switching the direction of the scale only shows a significant effect on means the first time the switch is performed. Subsequent switches do not affect measurement. This finding indicates that respondents most likely miss the fact of the switch. The analysis of response latencies and number of clicks showing no differences between the experimental groups supports this conclusion. For the second experiment, adding numerical labels demonstrates significant differences in means for the frequency items. Non-frequency items are not affected by the addition of numeric labels. Furthermore, no differences are found between starting the scale labels with 0 or starting with 1.
Originality/value: Sometimes it is needed to change the direction of the scale within the questionnaire or change the assignment of numeric labels (e.g., for replication purposes). Switching the scale directions and choosing the scale starting point might affect measurement. This study offers experimental evidence on both subjects. The findings that the first scale direction change and adding numeric labels in frequency scales affect respondents’ answers provide guidance for designing (web) questionnaires.
Research limitations/implications: This study demonstrates that the measurement is affected by switching the scale direction and the way of assigning numeric labels. In future studies, the focus should be on measurement error caused by such labeling schemes.
Practical implications: This study suggests that researchers should carefully consider the trade-offs when planning to use multiple scales and various numbering schemes within the same questionnaire.